Generative artificial intelligence responses to patient messages in the electronic health record: early lessons learned.

Background: Electronic health record (EHR)-based patient messages can contribute to burnout. Messages with a negative tone are particularly challenging to address. In this perspective, we describe our initial evaluation of large language model (LLM)-generated responses to negative EHR patient messages and contend that using LLMs to generate initial drafts may be feasible, although refinement will be needed. Methods: A retrospective sample (n = 50) of negative patient messages was extracted from a health system EHR, de-identified, and inputted into an LLM (ChatGPT). Qualitative analyses were conducted to compare LLM responses to actual care team responses. Results: Some LLM-generated draft responses varied from human responses in relational connection, informational content, and recommendations for next steps. Occasionally, the LLM draft responses could have potentially escalated emotionally charged conversations. Conclusion: Further work is needed to optimize the use of LLMs for responding to negative patient messages in the EHR.

AI-Generated Draft Replies Integrated Into Health Records and Physicians' Electronic Communication.

Importance: Timely tests are warranted to assess the association between generative artificial intelligence (GenAI) use and physicians' work efforts. Objective: To investigate the association between GenAI-drafted replies for patient messages and physician time spent on answering messages and the length of replies. Design, Setting, and Participants: Randomized waiting list quality improvement (QI) study from June to August 2023 in an academic health system. Primary care physicians were randomized to an immediate activation group and a delayed activation group. Data were analyzed from August to November 2023. Exposure: Access to GenAI-drafted replies for patient messages. Main Outcomes and Measures: Time spent (1) reading messages, (2) replying to messages, (3) length of replies, and (4) physician likelihood to recommend GenAI drafts. The a priori hypothesis was that GenAI drafts would be associated with less physician time spent reading and replying to messages. A mixed-effects model was used. Results: Fifty-two physicians participated in this QI study, with 25 randomized to the immediate activation group and 27 randomized to the delayed activation group. A contemporary control group included 70 physicians. There were 18 female participants (72.0%) in the immediate group and 17 female participants (63.0%) in the delayed group; the median age range was 35-44 years in the immediate group and 45-54 years in the delayed group. The median (IQR) time spent reading messages in the immediate group was 26 (11-69) seconds at baseline, 31 (15-70) seconds 3 weeks after entry to the intervention, and 31 (14-70) seconds 6 weeks after entry. The delayed group's median (IQR) read time was 25 (10-67) seconds at baseline, 29 (11-77) seconds during the 3-week waiting period, and 32 (15-72) seconds 3 weeks after entry to the intervention. The contemporary control group's median (IQR) read times were 21 (9-54), 22 (9-63), and 23 (9-60) seconds in corresponding periods. The estimated association of GenAI was a 21.8% increase in read time (95% CI, 5.2% to 41.0%; P = .008), a -5.9% change in reply time (95% CI, -16.6% to 6.2%; P = .33), and a 17.9% increase in reply length (95% CI, 10.1% to 26.2%; P < .001). Participants recognized GenAI's value and suggested areas for improvement. Conclusions and Relevance: In this QI study, GenAI-drafted replies were associated with significantly increased read time, no change in reply time, significantly increased reply length, and some perceived benefits. Rigorous empirical tests are necessary to further examine GenAI's performance. Future studies should examine patient experience and compare multiple GenAIs, including those with medical training.

Sustainable Approach to Justice, Equity, Diversity, and Inclusion Through Better Quality Measurement.

The US health care industry has broadly adopted performance and quality measures that are extracted from electronic health records and connected to payment incentives that hope to improve declining life expectancy and health status and reduce costs. While the development of a quality measurement infrastructure based on electronic health record data was an important first step in addressing US health outcomes, these metrics, reflecting the average performance across diverse populations, do not adequately adjust for population demographic differences, social determinants of health, or ecosystem vulnerability. Like society as a whole, health care must confront the powerful impact that social determinants of health, race, ethnicity, and other demographic variations have on key health care performance indicators and quality metrics. Tools that are currently available to capture and report the health status of Americans lack the granularity, complexity, and standardization needed to improve health and address disparities at the local level. In this article, we discuss the current and future state of electronic clinical quality measures through a lens of equity.

Benefits and Challenges of Transitioning Occupational Health to an Enterprise Electronic Health Record.

OBJECTIVE: Occupational health (OH) documentation has traditionally been separate from health system electronic health records (EHRs), but this can create patient safety and care continuity challenges. Herein, we describe outcomes and challenges of such integration including how one health system managed compliance with laws, regulations, and ethical principles concerning digital privacy. METHODS: Occupational health integration with the enterprise EHR at the University of California San Diego Health was started in June 2021 and completed in December 2021. RESULTS: Integrating with the enterprise EHR allowed for a secure telehealth system, faster visit times, digitization of questionnaires medical clearance forms, and improved reporting capabilities. CONCLUSIONS: Integration and interoperability are fundamental building blocks to any OH EHR solution and will allow for evaluation of worker population trends, and targeted interventions to improve worker health status.

Deep Dive into the Long Haul: Analysis of Symptom Clusters and Risk Factors for Post-Acute Sequelae of COVID-19 to Inform Clinical Care.

Long COVID is a chronic condition characterized by symptoms such as fatigue, dyspnea, and cognitive impairment that persist or relapse months after an acute infection with the SARS-CoV-2 virus. Many distinct symptoms have been attributed to Long COVID; however, little is known about the potential clustering of these symptoms and risk factors that may predispose patients to certain clusters. In this study, an electronic survey was sent to patients in the UC San Diego Health (UCSDH) system who tested positive for COVID-19, querying if patients were experiencing symptoms consistent with Long COVID. Based on survey results, along with patient demographics reported in the electronic health record (EHR), linear and logistic regression models were used to examine putative risk factors, and exploratory factor analysis was performed to determine symptom clusters. Among 999 survey respondents, increased odds of Long COVID (n = 421; 42%) and greater Long COVID symptom burden were associated with female sex (OR = 1.73, 99% CI: 1.16-2.58; ß = 0.48, 0.22-0.75), COVID-19 hospitalization (OR = 4.51, 2.50-8.43; ß = 0.48, 0.17-0.78), and poorer pre-COVID self-rated health (OR = 0.75, 0.57-0.97; ß = -0.19, -0.32--0.07). Over one-fifth of Long COVID patients screened positive for depression and/or anxiety, the latter of which was associated with younger age (OR = 0.96, 0.94-0.99). Factor analysis of 16 self-reported symptoms suggested five symptom clusters-gastrointestinal (GI), musculoskeletal (MSK), neurocognitive (NC), airway (AW), and cardiopulmonary (CP), with older age (ß = 0.21, 0.11-0.30) and mixed race (ß = 0.27, 0.04-0.51) being associated with greater MSK symptom burden. Greater NC symptom burden was associated with increased odds of depression (OR = 5.86, 2.71-13.8) and anxiety (OR = 2.83, 1.36-6.14). These results can inform clinicians in identifying patients at increased risk for Long COVID-related medical issues, particularly neurocognitive symptoms and symptom clusters, as well as informing health systems to manage operational expectations on a population-health level.

Population-Based Evaluation of Postacute Coronavirus Disease-2019 (COVID-19) Chronic Sequelae in Patients Who Tested Positive for Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2).

The true incidence and comprehensive characteristics of Long Coronavirus Disease-19 (COVID-19) are currently unknown. This is the first population-based outreach study of Long COVID within an entire health system, conducted to determine operational needs to care for patients with Long COVID.

Social Determinants of Health Data Availability for Patients with Eye Conditions.

Purpose: To quantify and characterize social determinants of health (SDoH) data coverage using single-center electronic health records (EHRs) and the National Institutes of Health All of Us research program. Design: Retrospective cohort study from June 2014 through June 2021. Participants: Adults 18 years of age or older with a diagnosis of diabetic retinopathy, glaucoma, cataracts, or age-related macular degeneration. Methods: For All of Us, research participants completed online survey forms as part of a nationwide prospective cohort study. In local EHRs, patients were selected based on diagnosis codes. Main Outcome Measures: Social determinants of health data coverage, characterized by the proportion of each disease cohort with available data regarding demographics and socioeconomic factors. Results: In All of Us, we identified 23 806 unique adult patients, of whom 2246 had a diagnosis of diabetic retinopathy, 13 448 had a diagnosis of glaucoma, 6634 had a diagnosis of cataracts, and 1478 had a diagnosis of age-related macular degeneration. Survey completion rates were high (99.5%-100%) across all cohorts for demographic information, overall health, income, education, and lifestyle. However, health care access (12.7%-29.4%), housing (0.7%-1.1%), social isolation (0.2%-0.3%), and food security (0-0.1%) showed significantly lower response rates. In local EHRs, we identified 80 548 adult patients, of whom 6616 had a diagnosis of diabetic retinopathy, 26 793 had a diagnosis of glaucoma, 40 427 had a diagnosis of cataracts, and 6712 had a diagnosis of age-related macular degeneration. High data coverage was found across all cohorts for variables related to tobacco use (82.84%-89.07%), alcohol use (77.45%-83.66%), and intravenous drug use (84.76%-93.14%). However, low data coverage (< 50% completion) was found for all other variables, including education, finances, social isolation, stress, physical activity, food insecurity, and transportation. We used chi-square testing to assess whether the data coverage varied across different disease cohorts and found that all fields varied significantly (P < 0.001). Conclusions: The limited and highly variable data coverage in both local EHRs and All of Us highlights the need for researchers and providers to develop SDoH data collection strategies and to assemble complete datasets.

Breast Cancer, Version 3.2022, NCCN Clinical Practice Guidelines in Oncology.

The therapeutic options for patients with noninvasive or invasive breast cancer are complex and varied. These NCCN Clinical Practice Guidelines for Breast Cancer include recommendations for clinical management of patients with carcinoma in situ, invasive breast cancer, Paget disease, phyllodes tumor, inflammatory breast cancer, and management of breast cancer during pregnancy. The content featured in this issue focuses on the recommendations for overall management of ductal carcinoma in situ and the workup and locoregional management of early stage invasive breast cancer. For the full version of the NCCN Guidelines for Breast Cancer, visit NCCN.org.

Development and implementation of an automated electronic health record-linked registry for emergency general surgery.

INTRODUCTION: Despite adoption of the emergency general surgery (EGS) service by hospitals nationally, quality improvement (QI) and research for this patient population are challenging because of the lack of population-specific registries. Past efforts have been limited by difficulties in identifying EGS patients within institutions and labor-intensive approaches to data capture. Thus, we created an automated electronic health record (EHR)-linked registry for EGS. METHODS: We built a registry within the Epic EHR at University of California San Diego for the EGS service. Existing EHR labels that identified patients seen by the EGS team were used to create our automated inclusion rules. Registry validation was performed using a retrospective cohort of EGS patients in a 30-month period and a 1-month prospective cohort. We created quality metrics that are updated and reported back to clinical teams in real time and obtained aggregate data to identify QI and research opportunities. A key metric tracked is clinic schedule rate, as we care that discontinuity postdischarge for the EGS population remains a challenge. RESULTS: Our registry captured 1,992 patient encounters with 1,717 unique patients in the 30-month period. It had a false-positive EGS detection rate of 1.8%. In our 1-month prospective cohort, it had a false-positive EGS detection rate of 0% and sensitivity of 85%. For quality metrics analysis, we found that EGS patients who were seen as consults had significantly lower clinic schedule rates on discharge compared with those who were admitted to the EGS service (85% vs. 60.7%, p < 0.001). CONCLUSION: An EHR-linked EGS registry can reliably conduct capture data automatically and support QI and research. LEVEL OF EVIDENCE: Prognostic and epidemiological, level III.

Digital Health Technologies for Remote Monitoring and Management of Inflammatory Bowel Disease: A Systematic Review.

INTRODUCTION: Digital health technologies may be useful tools in the management of chronic diseases. We performed a systematic review of digital health interventions in the management of patients with inflammatory bowel diseases (IBD) and evaluated its impact on (i) disease activity monitoring, (ii) treatment adherence, (iii) quality of life (QoL) measures, and/or (iv) health care utilization. METHODS: Through a systematic review of multiple databases through August 31, 2020, we identified randomized controlled trials in patients with IBD comparing digital health technologies vs standard of care (SoC) for clinical management and monitoring and reporting impact on IBD disease activity, treatment adherence, QoL, and/or health care utilization or cost-effectiveness. We performed critical qualitative synthesis of the evidence supporting digital health interventions in patients with IBD and rated certainty of evidence using Grading of Recommendations Assessment, Development and Evaluation. RESULTS: Overall, we included 14 randomized controlled trials (median, 98 patients; range 34-909 patients; follow-up <12 months) that compared web-based interventions, mobile applications, and different telemedicine platforms with SoC (clinic-based encounters). Although overall disease activity and risk of relapse were comparable between digital health technologies and SoC (very low certainty of evidence), digital health interventions were associated with lower rate of health care utilization and health care costs (low certainty of evidence). Digital health interventions did not significantly improve patients' QoL and treatment adherence compared with SoC (very low certainty of evidence). Trials may have intrinsic selection bias due to nature of digital interventions. DISCUSSION: Digital health technologies may be effective in decreasing health care utilization and costs, though may not offer advantage in reducing risk of relapse, QoL, and improving treatment adherence in patients with IBD. These techniques may offer value-based care for population health management.

A learning health system approach to COVID-19 exposure notification system rollout.

Introduction: Digital exposure notification (EN) approaches may offer considerable advantages over traditional contact tracing in speed, scale, efficacy, and confidentiality in pandemic control. We applied the science of learning health systems to test the effect of framing and digital means, email vs Short Message Service (SMS), on EN adoption among patients of an academic health center. Methods: We tested three communication approaches of the Apple and Google EN system in a rapid learning cycle involving 15 000 patients pseudorandomly assigned to three groups. The patients in the first group received a 284-word email that presented EN as a tool that can help slow the spread. The patients in the second group received a 32-word SMS that described EN as a new tool to help slow the spread (SlowTheSpreadSMS). Patients in the third group received a 47-word SMS that depicted the system as a new digital tool that can empower them to protect their family and friends (EmpowerSMS). A brief four-question anonymous survey of adoption was included in a reminder message sent 2 days after the initial outreach. Results: One hundred and sixty people responded to the survey within 1 week: 2.33% from EmpowerSMS, 0.97% from SlowTheSpreadSMS, and 0.53% from emails; 29 (41.43%), 24 (41.38%), and 11 (34.38%) reported having adopted EN from each group, respectively. Patient reported barriers to adoption included iOS version incompatibility, privacy concerns, and low trust of government agencies or companies like Apple and Google. Patients recommended that healthcare systems play an active role in disseminating information about this tool. Patients also recommended advertising on social media and providing reassurance about privacy. Conclusions: The EmpowerSMS resulted in relatively more survey responses. Both SMS groups had slightly higher, but not statistically significant EN adoption rates compared to email. Findings from the pilot not only informed operational decision-making in our health system but also contributed to EN rollout planning in our State.

NCCN Guidelines® Insights: Breast Cancer, Version 4.2021.

The NCCN Guidelines for Breast Cancer include up-to-date guidelines for clinical management of patients with carcinoma in situ, invasive breast cancer, Paget disease, phyllodes tumor, inflammatory breast cancer, male breast cancer, and breast cancer during pregnancy. These guidelines are developed by a multidisciplinary panel of representatives from NCCN Member Institutions with breast cancer-focused expertise in the fields of medical oncology, surgical oncology, radiation oncology, pathology, reconstructive surgery, and patient advocacy. These NCCN Guidelines Insights focus on the most recent updates to recommendations for adjuvant systemic therapy in patients with nonmetastatic, early-stage, hormone receptor-positive, HER2-negative breast cancer.

Medical Undistancing Through Telemedicine: A Model Enabling Rapid Telemedicine Deployment in an Academic Health Center During the COVID-19 Pandemic.

Background: The authors draw upon their experience with a successful, enterprise-level, telemedicine program implementation to present a "How To" paradigm for other academic health centers that wish to rapidly deploy such a program in the setting of the COVID-19 pandemic. The advent of social distancing as essential for decreasing viral transmission has made it challenging to provide medical care. Telemedicine has the potential to medically undistance health care providers while maintaining the quality of care delivered and fulfilling the goal of social distancing. Methods: Rather than simply reporting enterprise telemedicine successes, the authors detail key telemedicine elements essential for rapid deployment of both an ambulatory and inpatient telemedicine solution. Such a deployment requires a multifaceted strategy: (1) determining the appropriateness of telemedicine use, (2) understanding the interface with the electronic health record, (3) knowing the equipment and resources needed, (4) developing a rapid rollout plan, (5) establishing a command center for post go-live support, (6) creating and disseminating reference materials and educational guides, (7) training clinicians, patients, and clinic schedulers, (8) considering billing and credentialing implications, (9) building a robust communications strategy, and (10) measuring key outcomes. Results: Initial results are reported, showing a telemedicine rate increase to 45.8% (58.6% video and telephone) in just the first week of rollout. Over a 5-month period, the enterprise has since conducted over 119,500 ambulatory telemedicine evaluations (a 1,000-fold rate increase from the pre-COVID-19 time period). Conclusion: This article is designed to offer a "How To" potential best practice approach for others wishing to quickly implement a telemedicine program during the COVID-19 pandemic.

Barriers to Implementing an Artificial Intelligence Model for Unplanned Readmissions.

BACKGROUND: Electronic health record (EHR) vendors now offer "off-the-shelf" artificial intelligence (AI) models to client organizations. Our health system faced difficulties in promoting end-user utilization of a new AI model for predicting readmissions embedded in the EHR. OBJECTIVES: The aim is to conduct a case study centered on identifying barriers to uptake/utilization. METHODS: A qualitative study was conducted using interviews with stakeholders. The interviews were used to identify relevant stakeholders, understand current workflows, identify implementation barriers, and formulate future strategies. RESULTS: We discovered substantial variation in existing workflows around readmissions. Some stakeholders did not perform any formal readmissions risk assessment. Others accustomed to using existing risk scores such as LACE+ had concerns about transitioning to a new model. Some stakeholders had existing workflows in place that could accommodate the new model, but they were not previously aware that the new model was in production. Concerns expressed by end-users included: whether the model's predictors were relevant to their work, need for adoption of additional workflow processes, need for training and change management, and potential for unintended consequences (e.g., increased health care resource utilization due to potentially over-referring discharged patients to home health services). CONCLUSION: AI models for risk stratification, even if "off-the-shelf" by design, are unlikely to be "plug-and-play" in health care settings. Seeking out key stakeholders and defining clear use cases early in the implementation process can better facilitate utilization of these models.

A Path to Clinical Quality Integration Through a Clinically Integrated Network: The Experience of an Academic Health System and Its Community Affiliates.

INTRODUCTION: Academic medical centers (AMCs) and community physicians seeking to establish a clinically integrated network (CIN) may benefit from a road map to navigate the opportunities and challenges of such an organizational structure. Creating and participating in a CIN requires careful consideration, investment of time, financial resources, alignment of a new quality infrastructure, shared governance, and vision. POTENTIAL BENEFITS, CHALLENGES, AND REGULATORY CONSIDERATIONS: Potential AMC benefits include geographic clinical expansion, the ability to provide care for a broader population of patients, a mechanism to collaborate with regional physician graduates, and an expansion of available teaching sites for trainees. Potential benefits to community practices include propagation of high-value care, enhanced access to evidence-based protocols and priority measures, preparation for value-based reimbursement structures, and connection to an institution that produces future health care practitioners. Challenges to CIN creation include goal alignment, trust between AMC and community partners, acceptance of common quality measures and benchmarks, access to shared data, and local adoption of quality improvement activities. QUALITY AND INFORMATION TECHNOLOGY CONSIDERATIONS: At inception the mission was to create an innovative academic-community alliance delivering high-quality, high-value, personalized care. Defining the clinical quality goals, measurement, governance, and improvement strategy, as well as information technology structure and decision making, are described. FUTURE DIRECTIONS: The network continues to grow and now includes more than 350 physicians, in 16 different specialties across 50 different independent medical practices throughout Southern California. We believe this builds a firm foundation for value-based health care.

Relation of Statin Use Prior to Admission to Severity and Recovery Among COVID-19 Inpatients.

The impact of statins, angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers (ARBs) on coronavirus disease 2019 (COVID-19) severity and recovery is important given their high prevalence of use among individuals at risk for severe COVID-19. We studied the association between use of statin/angiotensin-converting enzyme inhibitors/ARB in the month before hospital admission, with risk of severe outcome, and with time to severe outcome or disease recovery, among patients hospitalized for COVID-19. We performed a retrospective single-center study of all patients hospitalized at University of California San Diego Health between February 10, 2020 and June 17, 2020 (n = 170 hospitalized for COVID-19, n = 5,281 COVID-negative controls). Logistic regression and competing risks analyses were used to investigate progression to severe disease (death or intensive care unit admission), and time to discharge without severe disease. Severe disease occurred in 53% of COVID-positive inpatients. Median time from hospitalization to severe disease was 2 days; median time to recovery was 7 days. Statin use prior to admission was associated with reduced risk of severe COVID-19 (adjusted OR 0.29, 95%CI 0.11 to 0.71, p < 0.01) and faster time to recovery among those without severe disease (adjusted HR for recovery 2.69, 95%CI 1.36 to 5.33, p < 0.01). The association between statin use and severe disease was smaller in the COVID-negative cohort (p for interaction = 0.07). There was potential evidence of faster time to recovery with ARB use (adjusted HR 1.92, 95%CI 0.81 to 4.56). In conclusion, statin use during the 30 days prior to admission for COVID-19 was associated with a lower risk of developing severe COVID-19, and a faster time to recovery among patients without severe disease.

A case study of the 1115 waiver using population health informatics to address disparities.

As participants in the California Medicaid 1115 waiver, the University of California San Diego Health (UCSDH) used population health informatics tools to address health disparities. This case study describes a modern application of health informatics to improve data capture, describe health disparities through demographic stratification, and drive reliable care through electronic medical record-based registries. We provide a details in our successful approach using (1) standardized collection of race, ethnicity, language, sexual orientation, and gender identity data, (2) stratification of 8 quality measures by demographic profile, and (3) improved quality performance through registries for wellness, social determinants of health, and chronic disease. A strong population health platform paired with executive support, physician leadership, education and training, and workflow redesign can improve the representation of diversity and drive reliable processes for care delivery that improve health equity.

Public Health Role of Academic Medical Center in Community Outbreak of Hepatitis A, San Diego County, California, USA, 2016-2018.

During 2016-2018, San Diego County, California, USA, experienced one of the largest hepatitis A outbreaks in the United States in 2 decades. In close partnership with local healthcare systems, San Diego County Public Health led a public health response to the outbreak that focused on a 3-pronged strategy to vaccinate, sanitize, and educate. Healthcare systems administered nearly half of the vaccinations delivered in San Diego County. At University of California San Diego Health, the use of informatics tools assisted with the identification of at-risk populations and with vaccine delivery across outpatient and inpatient settings. In addition, acute care facilities helped prevent further disease transmission by delaying the discharge of patients with hepatitis A who were experiencing homelessness. We assessed the public health roles that acute care hospitals can play during a large community outbreak and the critical nature of ongoing collaboration between hospitals and public health systems in controlling such outbreaks.

COVID-19 TestNorm: A tool to normalize COVID-19 testing names to LOINC codes.

Large observational data networks that leverage routine clinical practice data in electronic health records (EHRs) are critical resources for research on coronavirus disease 2019 (COVID-19). Data normalization is a key challenge for the secondary use of EHRs for COVID-19 research across institutions. In this study, we addressed the challenge of automating the normalization of COVID-19 diagnostic tests, which are critical data elements, but for which controlled terminology terms were published after clinical implementation. We developed a simple but effective rule-based tool called COVID-19 TestNorm to automatically normalize local COVID-19 testing names to standard LOINC (Logical Observation Identifiers Names and Codes) codes. COVID-19 TestNorm was developed and evaluated using 568 test names collected from 8 healthcare systems. Our results show that it could achieve an accuracy of 97.4% on an independent test set. COVID-19 TestNorm is available as an open-source package for developers and as an online Web application for end users (https://clamp.uth.edu/covid/loinc.php). We believe that it will be a useful tool to support secondary use of EHRs for research on COVID-19.

Breast Cancer, Version 3.2020, NCCN Clinical Practice Guidelines in Oncology.

Several new systemic therapy options have become available for patients with metastatic breast cancer, which have led to improvements in survival. In addition to patient and clinical factors, the treatment selection primarily depends on the tumor biology (hormone-receptor status and HER2-status). The NCCN Guidelines specific to the workup and treatment of patients with recurrent/stage IV breast cancer are discussed in this article.